PATENT APPLICATION 
METHOD AND SYSTEM FOR IMPROVING SCANNED IMAGE DETAIL 



CROSS REFERENCE TO RELATED APPLICATIONS 

This application claims the benefit of the following United States Provisional 
Patent Applications: Serial No. 60/234,520, filed on September 21, 2000, and entitled 
"Method of Generating an Image Mask for Improving Image Detail;" Serial No. 
60/234,408, filed on September 21, 2000, and entitled "Method of Applying An Image 
Mask For Improving Image Detail;" and Serial No. 60/285,591 filed on April 19, 2001, 
and entitled "Method and System and Software for Applying an Image Mask for 
Improving Image Detail;" of common assignee herewith. 

FIELD OF THE INVENTION 

The present invention relates generally to images and more particularly to 
improving the dynamic range of images. 

BACKGROUND OF THE INVENTION 

Images recorded on photographic film can represent a variety of dynamic ranges 
found in the subjects being recorded. Dynamic range is used to describe the variety of 
contrasts within an image. Dynamic range can include variations in intensity among a 
specific color, as well as variations in color. 

The dynamic range of a film negative is greater than the dynamic range available 
in a photographic print or in paper. When photographic film is being developed in a 
photo lab, the lab technician must determine which portion of the dynamic range to use 
from the negative. Generally, when features in one portion of the dynamic range are 
chosen, other details must be left out. 
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Areas of the image can be selectively chosen to be bring out detail in various 
portions of the image and to improve the detail when printing the image. This process 
can be long and must be performed on each different image. Image masks can be 
generated for selecting the portions of the dynamic range for best selecting the image 
5 detail. An image mask can be applied to a digitized image to alter the image contrasts 

and increase detail in areas that would normally be lost when converting the image to a 
photographic print or paper. Generating a mask can be performed automatically by 
generating an unsharp mask (USM). However, when the USM is applied to an image, 
the masked image that results generally has aberrations along edges. Areas around an 
10 edge within the masked image may contain undesirable portions of lightened or darkened 

*■ areas. Repairing the aberrations requires a technician to manually make changes to the 

masked image and can take several hours when handling multiple images. 
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SUMMARY OF THE INVENTION 

The present invention provides a method of generating an image mask for an 
original image, the method comprises generating a decimated representation of the 
original image. The method further includes applying a blurring algorithm to form a 
blurred representation of the decimated representation. The blurred representation 
includes a plurality of pixels having varying values to form sharp edges representative 
of rapidly changing boundaries in the decimated image. The method further includes 
combining the blurred representation with the decimated representation. 

An advantage of at least one embodiment of the present invention is that an image 
mask with sharp edges can be generated for improving the detail in a printed image. 
Another advantage of at least one embodiment of the present invention is that images 
with improved detail can be generated automatically. An additional advantage of at least 
one embodiment of the present invention is that calculations to improve the image detail 
in scanned images can be performed relatively quickly, due to a lower processing 
overhead less user intervention than conventional methods. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

Other objects, advantages, features and characteristics of the present invention, 
as well as methods, operation and functions of related elements of structure, and the 
combination of parts and economies of manufacture, will become apparent upon 
consideration of the following description and claims with reference to the accompanying 
drawings, all of which form a part of this specification, wherein like reference numerals 
designate corresponding parts in the various figures, and wherein: 

FIG. 1 is a block diagram illustrating a method for improving detail in a digital 
image, according to one embodiment of the present invention; 

FIG. 2 is a block diagram illustrating an original image and a blurred image is 
used for describing a method of performing a blurring algorithm on the original image, 
according to one embodiment of the present invention; 

FIG. 3 is a block diagram of a method for generating an image mask, according 
to at least one embodiment of the present invention; and 

FIG. 4 is a block diagram illustrating a method of applying an image mask to an 
image, according to at least one embodiment of the present invention. 
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DETAILED DESCRIPTION OF THE INVENTION 

Referring now to FIG. 1, a method of generating an image mask for improving 
image detail in a digitized image is shown. A mask of a scanned image is generated and 
applied to original image to generate a masked image C. The masked image C can be 
used to provide an image with improved image detail within a reproducible dynamic 
range. 

In step 1 10, an image is scanned to create a digital representation of an original 
image A. In one embodiment of the present invention, the image is recorded in 
photographic film. Once the photographic film has begun to develop, cameras can be 
used to generate an electronic representation (original image A) of the image stored in a 
negative of the photographic film. Recording the image directly from the negative can 
allow a representation to be taken using the high dynamic range available in the film 
negative. It will be appreciated that electronic representations may be taken of images 
recorded in other forms. For example, an image recorded in a photographic print can be 
scanned using an optical scanner to generate an electronic representation. Images may 
also be scanned from paper or magazine print. 

In step 120, an image mask B is generated from the original image A. The 
original image A, from step 1 10, and the generated image mask B are composed of 
pixels. Each pixel, has information about a particular point in the image. Each pixel can 
include information on the intensity or color associated with its point. Every pixel on the 
image mask B is generated relative to a pixel in the original image A. In a preferred 
embodiment of the present invention, the pixels generated for image mask B are 
calculated from averages among neighboring pixels in original image A. The 
calculations are based upon changes in contrast in original image A. 

Rapidly changing contrasts in original image A can be used to determine sharp 
edges in the generated image mask B. However, less rapidly changing contrast in original 
image A can be averaged to generate blurred regions in image mask B. The calculations 
performed on original image A can produce an image mask B which is capable of 
preserving the boundaries of original image A while blurring similar pixels, as will be 
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discussed further in FIG. 2. The generated image mask can be referred to as a sharp 
unsharp mask (SUM), maintaining sharp boundaries of unsharp regions. 

In step 130, the generated image mask B is applied to the scanned original image 
A. The original image A and the image mask B can be applied through an overlay. As 
discussed further in FIG. 4, a mathematical operation, including a division, between the 
pixel values of the original image A and the relative pixel values in the masked image B, 
is used to generate the masked image C. It will be appreciated that other functions for 
applying the masked image B to the original image A can be employed without departing 
from the scope of the present invention. 

In step 140, the masked image C can be printed. When masked image C is 
printed, a large portion of its dynamic range may be lost. By using image mask B to 
generate masked image C (step 130), the desirable detail in original image A can be 
preserved. In step 140, the masked image is ready for delivery in the form desired. The 
form in which the image may be delivered includes, but is not limited to, an electronic 
form, a photographic print, or a film record. Electronic outputs can be represented as a 
digital file, stored on mass storage devices, tape drives, CD recorders, or DVD recorders. 
Electronic outputs can also be transferred to other systems using a communications 
adapter, where the file can be sent to the Internet, an intranet, as an e-mail, etc. The 
output can also be displayed as an image on a display or printed using a computer printer. 
The image can also be prepared for retrieval at an image processing kiosk which allows 
customers to recover their pictures and print them out in a form of their choosing without 
the assistance of a film development technician. Furthermore, the image can be 
represented on a form of film record, such as a film negative or positive image. 

Referring now to FIG. 2, a diagram of an original image and a blurred image are 
shown, according to one embodiment of the present invention. Original image 210, 
composed of 25 pixels, such as pixels 215-219, is used to generate blurred image 230, 
composed of 25 pixels, such as pixels 235-239. A blurring algorithm can be used to 
maintain sharp edges among rapidly changing contrasts, while blurring regions of less 
rapidly changing contrasts. 
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An original image 210 can be an electronic representation of an image scanned 
from film, scanned from print, or stored electronically in some form, such as on a 
computer disk or in a mass storage device. Original image 210 can contain detail which 
may not be appropriately represented when output to the display or in print. To improve 
the detail in image 210, the details must be brought out, in contrast to surrounding 
portions of the image. Image contrast in shadows must be high to brighten out detail in 
the dark regions of shadowed areas. However, simply increasing the image contrast 
could destroy detail found in highlighted areas. Highlighted areas must have the image 
contrast reduced to darken less lightened sections and bring out the details in areas which 
are almost overexposed on photographic film. However, simply dividing film into a 
highlight area and a shadow area will not suffice to improve complex images. Images 
can contain complex and diverse regions of varying contrast levels. To properly handle 
these images, a blurred image B can be generated. 

In one embodiment of the present invention, pixels calculated for blurred image 
230 relate to averages taken over regions of pixels in original image 210. For example, 
pixel 233 relates to calculations performed around pixel 213. In one embodiment, pixels 
21 1, 212, 214, and 215 are used to determine the averaging that should be performed on 
pixel 213 to generate pixel 233. Pixel 213 can be seen as a central pixel, the value of 
which can be changed dependant on the values of its surrounding pixels. However, 
simply averaging the values of pixels 211-215 to determine pixel 233 is not adequate. 

Averaging all the pixel values would simply create a blur of original image 210. 
Rapid changes in contrast among the pixels in original image 210 should be preserved 
as sharp edges in blurred image 230. To preserve sharp edges, changes in contrast must 
be accounted for. For example, if a sharp change in contrast is detected between pixels 
213 and 214, the values of pixels 213 and 214 should not be averaged together, allowing 
the difference in contrast to be preserved as an edge. Furthermore, distance from the 
center pixel should also be a determining factor in calculating pixels in blurred image 
230. If, for example, too large an area is considered, little blurring may result. If original 
image 210 contains a large variety of contrasts, a central pixel being considered, such as 
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pixel 213, may not be altered by much since there may exist enough of a variety of pixels 
with intensities above and below its own intensity value. 

Several factors must be considered in creating blurred image 230. Distance from 
the central pixel must be small enough to not average in too many variations in contrast; 
yet the distance must be large enough to create an adequate blurring of the central pixel. 
Accordingly, the pixels farther from the central pixel should be given less weight in 
averaging their values with the central pixel. For example, pixels 215 and 21 1 should be 
given less weight in determining the value of pixel 233, because they are farther away 
from the central pixel, pixel 213, than pixels 212 and 214. Furthermore, rapidly changing 
contrasts should also be accounted for. Accordingly, a function can be used to determine 
the weight of pixel values in original image 210 for determining pixel values for blurred 
image 230. The weighting equation can be described by the following equation: 



The weight function, w N , can be used to apply a separate weight to each of the pixel 
values. If the value of w N is returned as a negative value, the returned weight for the 
pixel being weighed can be zero. For example, if pixel 233 was being calculated, w N 
could be used to apply a weight to each of the pixels 21 1-215. PixelN is the contrast 
value of the pixel being weighed. Centerpixel is the contrast value of the central pixel, 
around which the blurring is being performed. For example, if pixel 236 is being 
calculated, the central pixel is pixel 216. Gain is a threshold value used to determine a 
contrast threshold for a sharp edge. For example, if pixel 233 is being calculated, the 
central pixel is 213. If the difference in contrast between pixel 213 and pixel 214 is 15, 
and Gain is set to 10, the returned value of w N is negative. Accordingly, in this example, 
pixel 214 is assigned a weight of zero, keeping the value of pixel 214 from affecting the 
averaging of pixel 213. 
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The value of Gain can be decreased as the pixel being weighed is further from the 
central pixel. Lowering the value of Gain allows small changes in the contrast between 
pixelN and centerpixel to result in negative w N , and thus be weighed to zero. 
Accordingly, the farther the pixel is from the centerpixel, the smaller Gain gets and the 
more likelihood, that the pixel will be assigned a weight of zero. The choice of gain is 
chosen to preferably decrease slowly as the distance from the central pixel is increased. 
It has been found that slower changes in Gain provide images with more pleasing detail 
than sharper changes in Gain. However, the values of Gain used can be adapted for the 
desired application. Furthermore, the weight function itself can be altered without 
departing from the scope of the present invention. 

Once the weights of the surrounding pixels have been calculated, a sum of each 
of the pixel values (multiplied by their relative weights) can be calculated. The sum can 
then be divided by the sum of the individual weights to generate the weighted average of 
the pixels, which can be used for the pixels of blurred image 230. The minimum weight 
calculated from the pixels adjacent to the central pixel can also be used and multiplied 
by each of the pixels surrounding the central pixel. Multiplying the weight of the 
adjacent pixel allows the blurring to be effectively turned 'off if the contrast around the 
central pixel is changing too rapidly. For example, if the difference in contrast strictly 
warrants a sharp edge in blurred image 230, the weight should be close to zero, forcing 
all other values to zero and allowing the central pixel to retain its value, effectively 
creating a sharp edge in blurred image 230. 

The processes performed to generate blurred image 230 can be likened to a 
sandblaster. A sandblaster can be used to soften, or blur, the textures it is working over. 
A sandblaster has an effective radius in which it is used over, with the material closer to 
the center of the sandblasting radius affected most. In the blurring algorithm described, 
a radius can be selected from the central pixel. The pressure of a sandblaster can be 
adjusted to affect more change. The Gain value in the described algorithm can be altered 
to affect more or less blurring. Accordingly, the blurring algorithm used will be herein 
referred to as the sandblaster algorithm. 
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The sandblaster algorithm can be performed in one dimensional increments. For 
example, to calculate the value of pixel 237, the pixels surrounding pixel 217 are 
considered. In one embodiment of the present invention, the averaged pixel values are 
determined using the neighboring vertical pixels and then the neighboring horizontal 
pixel values. For example, when calculating pixel 233, the pixels surrounding pixel 213 
are preferably considered. Pixels 211, 212, 214, and 215 would initially be used to 
average the value of pixel 213 for pixel 233. Then the values of pixels 216, 217, 218, 
and 219 would be taken into account. Alternatively, windows can be generated and 
applied to average in pixels around the central pixel together, in both the horizontal and 
vertical directions. Color images can be composed of multiple image planes, wherein the 
multiple image planes may include planes for each color, a red plane, a green plane, and 
a blue plane. In a preferred embodiment, the sandblaster algorithm is only performed on 
one plane at a time. Alternatively, the sandblaster algorithm can be calculated taking 
other image planes into account, calculating in the values of pixels relative to the central 
pixel but in a different color plane. However, it should be noted that by performing such 
multi-dimensional calculation over an image may increase the processing overhead. 
Additionally, pixels which are near an image edge, such as image 216 may ignore values 
desired from pixels beyond the limits of original image 210. In one embodiment, the 
images along the edge use their value to reproduce pixels values beyond the image edge, 
for calculation of the sandblaster algorithm. Additionally, zeroes may be taken for values 
lying outside original image 210. Furthermore, a wraparound effect may be employed, 
wherein the values of pixels on the other side of original image 210 are used. For 
example, the value of pixel 219 may be used in considering the neighbors of pixel 216. 

Referring now to FIG. 3, a block diagram of a method for generating an image 
mask is shown, according to at least one embodiment of the present invention. As 
previously discussed, the sandblaster blurring algorithm can be used to create a blurred 
image with sharp edges and blurred regions. To improve the detail captured by an image 
mask incorporating the sandblaster blurring algorithm, a pyramidal decomposition is 
performed on the original image, as shown in FIG. 3. 
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In step 310, the original image is received. As previously discussed, the original 
image is a digital image representation of an image, taken through scanning or other 
methods. In step 335, the image size is reduced. In at least one embodiment, the image 
size is reduced in half. The reduction in image size may be performed using a standard 
digital image decimation. In one embodiment, the decimation is performed by discarding 
every other pixel in the original image from step 310. 

In step 325, the sandblaster blurring algorithm, as discussed in FIG. 2, is 
performed on the decimated image to create a blurred image. In step 336, the blurred 
image is decimated once again. In step 326, the decimated image from step 336 is 
blurred using the sandblaster blurring algorithm. Further decimation steps 337-339 and 
sandblaster blurring steps 327-329 are consecutively performed on the outputs of 
previous steps. In step 350, the blurred image from the sandblaster step 329 is subtracted 
from the decimated output from the decimation step 350. In step 360, the mixed output 
from step 350 is up-sized. In one embodiment, the image is increased to twice its pixel 
resolution. Increasing the image size may be performed by simply repeating the image 
values present for new pixels. Pixel interpolation may also be performed to determine 
the values of the new pixels, in step 352, the up-sized image from step 360, is added to 
the blurred image from step 328. The combined image information is subtracted from 
the decimated output from step 338. The calculations in step 352 are performed to 
recover image detail that may have been lost. Mixer steps 354 and 352, consecutively 
performed with up-size steps 362-366, attempt to generate mask data. In step 358, a 
mixer is used to combine the up-sized image data from step 366 with the blurred image 
data from step 325. The output from the mixer in step 358 is then up-sized, in step 380, 
to produce the image mask of the received image. The image mask is then prepared for 
delivery and use, as in step 390. 

It will be appreciated that additional or less blurring may be performed among the 
steps of the pyramidal decomposition described herein. It should be noted that by not 
performing the blurring algorithm on the original image, much processing time may be 
reduced. Calculations based on the decimated images can be performed faster and with 
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less overhead than calculations based off the original image, producing detailed image 
masks. The image masks produced using the described method preferably include sharp 
edges based on rapidly changing boundaries found in the original image, and blurred 
regions among less rapidly changing boundaries. 

In the described embodiment, pyramidal decomposition is performed along a 
single image color plane. It will be appreciated that additional color planes may also be 
presented in the steps shown. Furthermore, information from different color planes may 
be processed concurrently. According to at least one embodiment of the present 
invention, the resultant image mask generated is a monochrome mask, used to apply itself 
to the intensities of the individual image color planes in the original image. A 
monochrome image plane can be calculated from separate image color planes. For 
example, in one embodiment, the values of the monochrome image mask are determined 
using the following equation: 

OUT = MAX(R,G). 

OUT refers to the pixel of the monochromatic image mask. MAX(R,G) is a function in 
which the maximum intensity between the intensity value of the pixel in the red plane and 
the intensity value of the pixel in the green plane is chosen. In the case of a mask pixel 
which contains more than 80% of its intensity from the blue plane, the formula can be 
appended to include: 

OUT = OUT + 50%B. 

wherein 50%B is half of the intensity value in the blue plane. The image mask may also 
be made to represent image intensities, such as the intensity among black and white 
values. It will be appreciated that while full color image masks may be used, they will 
require heavier processing overhead. 
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Referring now to FIG. 4, a method for generating a masked image is shown, 
according to at least one embodiment of the present invention. Image information related 
to an original image 210 is mathematically combined with intensity information from an 
image mask 230. The combined data is used to create a masked image 410, which 
contains improved reproducible image detail over the original image 210. 

Image data to generate masked image 410 can be calculated on a pixel by pixel 
basis. Each corresponding pixel from original image 210 and image mask 230 is 
combined to form a pixel on masked image 410. For example, pixel data from pixel 215, 
of original image 210, is combined with pixel information from pixel 235, of image mask 
230, using mathematical manipulation, such as overlay function 420. The combined data 
is used to represent pixel 415 of masked image 410. 

Overlay function 420 is a function used to overlay the pixel information between 
original image 210 and image mask 230. In one embodiment of the present invention, 
overlay function 420 involves mathematical manipulation and is defined by the equation: 

— MASK + — 
4 4 

OUT refers to the value of the pixel in masked image 410. IN refers to the value of the 
pixel taken from original image 210. MASK refers to the value of the corresponding 
pixel in image mask 230. For example, to produce the output value of pixel 414, the 
value of pixel 214 is divided by 3/4 the value of pixel 234, added with an offset. The 
offset, 1/4, is chosen to prevent an error from occurring due to diving by zero. The offset 
can also be chosen to lighten shadows in the resultant masked image 410. 

As previously discussed, image mask 230 can be a monochromatic mask. The 
mask can be used to control the white and black levels in images. Grayscale contrast is 
the contrast over large areas in an image. Image contrast can be used to refer to the 
contrast of details within an image. Through manipulation of overlay function 420, the 
grayscale contrast and the image contrast can be altered to best represent original image 
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210. In one embodiment of the present invention, overlay function 420 is altered 
according to settings made by a user. Independent control of the image contrast and 
grayscale contrast can be provided. Control can be used to produce images using low 
image contrast in highlights and high image contrast in shadows. Additionally, functions 
can be added to control the generation of the mask. Control can be offered over the 
pressure (Gain) and radius (region) effected through the sandblaster blurring algorithm 
(described in FIG. 2). Additionally, control over the histogram of the image can be 
offered through control over the image contrast and the grayscale contrast. A normalized 
image can be generated in which histogram leveling can be performed without destroying 
image contrast. The controls, functions, and algorithms described can be performed 
within an information handling system. It will be appreciated that other systems may be 
employed, such as through image processing kiosks, to produce masked image 410, in 
keeping with the scope of the present invention. 

Although an embodiment of the present invention has been shown and described 
in detail herein, along with certain variants thereof, many other varied embodiments that 
incorporate the teachings of the invention may be easily constructed by those skilled in 
the art. Accordingly, the present invention is not intended to be limited to the specific 
form set forth herein, but on the contrary, it is intended to cover such alternatives, 
modifications, and equivalents, as can be reasonably included within the spirit and scope 
of the invention. 
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